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Executive  Summary 

The  Operational  Safety,  Suitability,  and  Effectiveness  (OSS&E)  Execution  Plan  as  a  whole  is  not 
suitable  to  be  used  as  a  Systems  Engineering  Plan  (SEP).  The  reason  for  this  is  the  OSS&E  Execution 
Plan  shows  how  a  program  will  develop  a  disciplined  systems  engineering  process,  while  the  SEP 
shows  what  that  disciplined  systems  engineering  process  is  and  how  it’s  being  implemented  on  the 
program. 

Though  the  execution  plan  is  not  suitable  as  a  SEP,  the  information  generated  through  OSS&E  thinking 
and  products  captures  systems  engineering  planning  that  should  be  incorporated  into  the  SEP.  As  the 
figure  below  shows,  an  OSS&E  Level  6  program  should  have  enough  systems  engineering  information 
to  partially  satisfy  nine  SEP  paragraphs  and  completely  satisfy  four  SEP  paragraphs  as  described  in  the 
OUSD(AT&L)  Systems  Engineering  Plan  Preparation  Guide,  version  1.02. 


SEP  Paragraph  Status  After  Applying  OSS&E 
Generated  Information 


Legend 

I  OSS&E  generated  information  can  satisfy  SEP  paragraph 
I  OSS&E  generated  information  can  partially  satisfy  SEP  paragraph 
M  OSS&E  generated  information  doesn’t  help  satisfy  SEP  paragraph 


The  three-digit  paragraph  numbers  in  the  above  figure  correspond  to  bulleted  subparagraphs  as  shown  in 
the  “Suggested  SEP  Format”  within  the  OUSD(AT&L)  Systems  Engineering  Plan  Preparation  Guide. 

A  sustainment  program  can  exploit  the  existing  systems  engineering  planning  done  in  support  of 
OSS&E  and  fill  in  the  gaps  with  other  program  documentation  and  some  additional  planning. 

The  analysis  shows  that  the  OSS&E  generated  information  from  each  OSS&E  level  can  be  incorporated 
to  meet  SEP  requirements,  but  that  additional  information  will  also  be  necessary.  Recommendations  on 
how  to  fill  the  OSS&E  gaps  for  a  sustainment  SEP  are  included  within  the  report. 


Air  Force  Center  for  Systems  Engineering  at  AFIT 
AFIT/SYA 

1050  Hobson  Way,  Bldg.  641,  Room  327 
Wright-Patterson  AFB,  OH  45433-7765 
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OSS&E  Planning  to  SEP  Preparation  Guidance  Gap  Analysis 

1.  Introduction 

Operational  Safety,  Suitability,  and  Effectiveness  (OSS&E)  is  a  process  intended  to  assure  engineering 
rigor  and  discipline  is  applied  during  weapons  system  sustainment,  the  operations  and  support  phase  of 
the  Integrated  Defense  Acquisition,  Technology,  and  Logistics  Life  Cycle  Management  Framework.  The 
Systems  Engineering  Plan  (SEP)  is  intended  to  guide  the  systems  engineering  effort  throughout  a 
weapon  system’s  life  cycle  and  to  show  the  Milestone  Decision  Authority  (MDA)  and  Program 
Executive  Officer  (PEO)  that  disciplined  systems  engineering  processes  and  practices  are  in  place  and 
used.  OSS&E  and  the  Systems  Engineering  Plan  are  both  tools  intended  to  invigorate  systems 
engineering  in  support  of  safe,  suitable,  and  effective  weapon  systems. 

In  support  of  the  release  of  the  new  AFI  63-1201,  Systems  Engineering ,  SAF/AQR  requested  the  Air 
Force  Center  for  Systems  Engineering  at  AFIT  to  provide  “a  Gap  Analysis  between  the  AFMCI  63-1201 
requirements  for  an  Operational  Safety,  Suitability,  and  Effectiveness  (OSS&E)  Assurance  Plan  and  the 
OSD  SEP  Guide  requirements.” 

This  document  was  also  written  to  be  released  at  the  same  time  and  with  the  updated  AFI  63-1201, 
Systems  Engineering  and  the  new  AFMCI  63-1201.  This  updated  AFI  supersedes  the  older  version 
titled  Operational  Safety,  Suitability,  and  Effectiveness.  The  new  AFI  version  doesn’t  eliminate 
OSS&E,  but  includes  OSS&E  as  part  of  the  bigger  systems  engineering  effort. 


1.1  Purpose 

This  report  has  five  purposes: 

a.  Document  how  OSS&E  efforts  are  directly  related  to  the  Systems  Engineering  Plan. 

b.  Show  the  gaps  that  exist  between  OSS&E  and  Systems  Engineering  Plan  requirements. 

c.  Provide  guidance  on  what  information  is  needed  to  fill  the  gaps. 

d.  Capture  the  analysis  that  led  to  the  gap  identification  and  sustainment  guidance. 

e.  Be  useful  for  sustainment  programs  developing  a  Systems  Engineering  Plan. 


1.2  Why  Write  a  Systems  Engineering  Plan ? 

So,  why  write  a  Systems  Engineering  Plan ?  Didn’t  Dwight  D.  Eisenhower  once  say,  “In  preparing  for 
battle  I  have  always  found  that  plans  are  useless,  but  planning  is  indispensable”?  The  simple  truth  of  the 
matter  is  we  write  SEPs  to  capture  the  systems  engineering  planning  we’ve  done.  The  not-so-simple, 
not-so-hidden  agenda  is  to  allow  the  writing  process  to  help  guide  and  expand  on  that  planning  by 
giving  us  a  chance  to  find  and  fill  gaps  in  our  indispensable  planning. 
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Why  do  we  need  to  capture  our  systems  engineering  planning?  There  are  at  least  four  very  valid 
reasons: 

a.  To  remember  what  we  decided  to  do.  Ever  think,  “Okay,  what’s  next?”  Chances  are  “what’s 
next”  for  systems  engineering  was  already  decided  on  and  may  have  even  been  documented 
somehow.  At  the  very  least  your  SEP  should  have  a  pointer  to  that  documentation  or,  perhaps, 
your  SEP  will  be  that  documentation. 

b.  To  assure  continuity.  Ever  think,  “Well,  how’d  we  do  it  last  time?”  For  recurring  activities, 
your  SEP  should  point  to  or  contain  that  information  as  well  as  the  success  or  failure  of  “how  we 
did  it  last  time”.  There’s  no  need  to  reinvent  the  wheel  every  time  you  need  one  and  there’s  no 
need  to  reproduce  a  square  wheel  that  just  doesn’t  work  -  but,  maybe  if  we  chip  away  at  the 
comers  ...  ? 

c.  To  train  new  people:  Ever  think,  “I  wish  I  didn’t  have  to  explain  everything  to  the  new  guy”? 
With  a  SEP  you  no  longer  have  to.  Be  bold.  Tell  the  new  guy  to  set  up  a  Test  Readiness 
Review.  The  new  guy  should  be  able  to  go  to  SEP  paragraph  2.4.2,  Technical  Review  Planning ; 
select  the  plan  on  accomplishing  a  Test  Readiness  Review;  and  begin  putting  one  together.  Of 
course,  the  new  guy  will  still  need  guidance  and  clarification,  but  hey,  the  new  guy  is  new. 

d.  To  show  the  high-level  decision  makers  we  know  what  we’re  doing.  Ever  think,  “I  hope  we 
know  what  we’re  doing”?  Well,  the  Program  Executive  Officers  (PEO)  and  Milestone  Decision 
Authorities  (MDA)  are  always  thinking  that.  Your  SEP  is  your  chance  to  show  your  PEO  or 
MDA  that  you  know  what  you’re  doing  so  that  the  PEO  or  MDA  can  know,  “We  know  what 
we’re  doing.” 

Notice  there’s  nothing  about  doing  business  better  or  saving  time  and  money  in  the  reasons  to  capture 
our  systems  engineering  planning.  Those  things  are  why  we  do  the  planning. 

Why  is  it  so  hard  to  write  a  SEP?  Could  it  be  because  technical  people  tend  to  process  information 
visually  but  try  to  explain  information  in  a  SEP  verbally?  Could  it  be  because  we  don’t  know  where  our 
planning  has  been  captured?  Could  it  be  we  try  to  fill  in  all  the  perceived  holes  to  make  a  perfect 
document  when  reality  is  more  than  adequate?  Could  it  be  that  the  person  writing  the  SEP  has  had  no 
exposure  to  the  planning?  Could  it  be  we  are  writing  a  textbook  on  systems  engineering  instead  of 
describing  how  we  do  systems  engineering?  The  bottom  line  here  is  a  SEP  documents  the  who,  what, 
when,  where,  why,  and  how  of  your  systems  engineering  implementation  that  was  identified  during  the 
planning.  Planning  is  only  hard  because  we  make  it  that  way.  Dwight  D.  Eisenhower  alluded  to  this 
when  he  said,  “An  intellectual  is  a  man  who  takes  more  words  than  necessary  to  tell  more  than  he 
knows.” 

The  hard  part  of  writing  a  Systems  Engineering  Plan  is  doing  the  planning.  Documenting  the  plan  may 
be  time  consuming  but  executing  the  plan  should  be  easy. 
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1.3  Instructions  on  How  to  Use  This  Document 

This  document  is  not  supposed  to  be  fancy  and  contains  no  magic  for  automatic  SEP  success.  What  this 
document  contains  is  information  you  can  use  to  see  how  OSS&E  efforts  relate  to  the  SEP  and  guidance 
that  can  get  you  going  with  sustainment  SEP  development.  The  real  meat  of  this  document  is  contained 
in  the  appendices.  So: 

a.  If  you  want  to  see  how  the  OSS&E  levels  relate  to  the  SEP: 

Read  paragraph  2.1  and  look  at  Appendix  C,  OSS&E  Level  to  SEP  Traceability  Matrix. 

b.  If  you  want  to  see  the  SEP  gaps  after  applying  OSS&E  developed  material: 

Read  paragraph  2.2,  Finding  the  Connections  -  Mapping  OSS&E  to  the  SEP. 

c.  If  you  want  to  write  a  sustainment  SEP: 

Use  the  OUSD(AT&L)  Defense  Systems,  Systems  Engineering  Plan  Preparation  Guide, 
version  1 .02,  as  a  guide  for  overall  format  and  content. 

Use  paragraph  2.2,  Finding  the  Connections  -  Mapping  OSS&E  to  the  SEP,  and 
Appendix  C,  OSS&E  Level  to  SEP  Traceability  Matrix,  as  a  guide  to  help  reuse 
information  generated  as  part  of  the  OSS&E  effort. 

d.  If  you  want  to  see  what  material  was  researched  in  the  analysis  and  writing  of  this  report: 

See  Appendix  B,  Documents  Referenced  and  Researched. 

e.  If  you  have  questions  or  comments  or  would  like  clarification: 

Contact  Steven  Pavick 

Air  Force  Center  for  Systems  Engineering 
AFIT/SYA 

2950  Hobson  Way,  Bldg.  641,  Room  327 
Wright-Patterson  AFB,  OH  45433-7765 

937  255-3355  x3368  or  DSN  785-3355  x3368 
steven.pavick@afit.edu  or  steven.pavick@afit.af.mil 
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2.  Analysis  and  Results 

2.1  Mapping  Requirements  -  OSS&E  to  SEP  Requirements  Tracing 

In  the  HQ  AFMC/DR/EN  Memo,  Execution  of  Air  Force  Polices  for  Assurance  of  Operational  Safety, 
Suitability,  and  Effectiveness  (OSS&E),  25  September  2000,  six  levels  of  OSS&E  implementation  are 
identified  including  the  development  of  an  OSS&E  Execution  Plan  as  part  of  level  3.  The  execution 
plan  was  intended  for  programs  to  show  how  they  would  get  through  level  6,  not  how  programs  would 
do  OSS&E.  That  how  to  do  OSS&E  falls  out  from  the  thinking  and  products  generated  to  meet  the  exit 
criteria  for  each  OSS&E  level.  It  is  the  information  captured  in  the  products  that  can  be  transferred  to 
the  Systems  Engineering  Plan.  The  OSS&E  levels  and  a  summary  of  each  level’s  requirements  are 
shown  in  table  1 . 

Table  1:  Summary  of  OSS&E  Levels  and  Requirements 


OSS&E  Level  and  Title 

Requirements/Exit  Criteria  Summary 

Level  1  -  Chief  Engineer  Assigned 

A  person  is  assigned  as  the  Chief  Engineer  for  a  specific 
system/end-item. 

Level  2  -  Configuration  Control 
Process  Established 

A  configuration  control  process  is  documented  and  actually 
in  use.  Authorities  within  the  process  are  documented  and 
assigned.  Configuration  control  training  is  established. 

Level  3  -  Plan  to  Assure  and 

Preserve  OSS&E  Documented 

Identify  how  the  system/end-item’s  program  will  attain 
OSS&E  Level  6  -  Full  OSS&E  Policy  Compliance. 

Level  4  -  OSS&E  Baselines 
Developed  and  Coordinated  with 

User 

Critical  system  attributes  known  as  OSS&E  Baseline 
Characteristics  are  documented  and  coordinated  with  the 
system/end- item  user. 

Level  5  -  OSS&E  Assessment  of 
Fielded  Systems/End-Items 

Actual  use  and  maintenance  data  is  collected  from  fielded 
systems/end-items.  The  data  is  analyzed  and  compared  to 
the  OSS&E  Baseline  Characteristics.  If  disconnects  exist, 
then  either  the  characteristics  are  modified  or  the  fielded 
systems/end-items  are  fixed. 

Level  6  -  Full  OSS&E  Policy 
Compliance 

A  disciplined  systems  engineering  process  should  be  in  place 
with  feedback  built  in  to  monitor  OSS&E  health. 

Information  generated  in  meeting  each  OSS&E  level  exit  criteria  can  be  used  to  partially  satisfy  the 
criteria  of  one  or  more  major  SEP  paragraphs.  Unfortunately,  OSS&E  generated  information  won’t 
fully  satisfy  any  of  the  major  SEP  paragraphs.  The  high  level  relationship  mapping  from  OSS&E  level 
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to  major  SEP  paragraph  (as  shown  in  the  OUSD(AT&L)  Systems  Engineering  Plan  Preparation 
Guide,  “Suggested  SEP  Format”)  is  shown  in  Figure  1.  The  details  behind  the  figure  are  included  in 
Appendix  C,  OSS&E  Level  to  SEP  Traceability  Matrix. 


Figure  1:  OSS&E  Level  to  SEP  Major  Paragraph  Mapping 

Figure  1  shows  that  if  a  program  is  OSS&E  level  1  or  higher,  it  has  already  generated  information  that 
could  be  used  to  address  the  requirements  of  SEP  paragraphs  1.1,  Program  Description  and  Applicable 
Documents,  and  2.2,  SE  Organizational  Integration  and  Technical  Authority.  Combine  this  with 
Appendix  C,  OSS&E  Level  to  SEP  Traceability  Matrix,  and  you’ll  find  the  applicable  information  is  the 
chief  engineer’s  name  and  how  the  program  interfaces  with  the  System/End-Item  (S&EI)  List. 

The  requirements  from  each  OSS&E  level  continue  to  add  to  the  information  that  can  carry  to  a  SEP, 
but  even  a  level  6  program  hasn’t  generated  enough  information  through  their  OSS&E  efforts  to 
completely  satisfy  any  of  the  major  SEP  paragraphs. 
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2.2  Finding  the  Connections  -  Mapping  OSS&E  to  the  SEP 

To  figure  out  how  information  created  for  OSS&E  implementation  can  be  used  to  fulfill  the  requirement 
for  a  SEP,  the  tables  in  appendix  C,  OSS&E  Level  to  SEP  Traceability  Matrix,  were  inverted.  This 
inversion  allows  us  to  see  how  the  OSS&E  generated  information  applies  directly  to  the  SEP.  Table  2 
shows  the  inverted  matrix  with  the  relationship  to  a  third  SEP  level.  This  third  SEP  level  corresponds 
to  the  bulleted  subparagraphs  as  shown  in  the  “Suggested  SEP  Format”  within  the  OUSD(AT&L) 
Systems  Engineering  Plan  Preparation  Guide. 

Table  2  shows  an  OSS&E  level  5  program  has  all  the  information  needed  for  SEP  paragraphs  2.1.2,  Key 
Performance  Parameters,  and  2.4.1,  Technical  Baseline  Management  and  Control.  A  level  6  program 
has  the  additional  information  needed  to  also  meet  the  SEP  paragraph  2.1.4,  Certification  Requirements, 
and  2.3.1,  Process  Selection,  requirements  fully.  No  other  areas  of  the  SEP  are  more  than  partially 
fulfilled  through  the  use  of  OSS&E  generated  information.  Further  detail  can  be  seen  in  Appendix  D, 
OSS&E  Level  to  SEP  Paragraph  Mapping. 

2.2.1  Complete  Connections 

What  follows  is  the  reasoning  behind  the  conclusion  that  an  OSS&E  Level  6  program  can  fully  satisfy 
four  SEP  paragraphs:  2.1.2,  Key  Performance  Parameters’,  2.1.4,  Certification  Requirements',  2.3.1, 
Process  Selection',  and  2.4.1,  Technical  Baseline  Management  and  Control.  Since  each  program  is  free 
to  implement  and  document  OSS&E  as  it  makes  sense  for  the  program,  specific  OSS&E  artifacts  are  not 
mentioned.  This  section  lists  the  applicable  OSS&E  level  exit  criteria  for  which  useful  artifacts  should 
have  been  developed. 
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Table  2:  SEP  Status  After  Applying  OSS&E  Generated  Information 


SEP 

Paragraph 

Number 

SEP  Paragraph  Title 

OSS&E 

Level  1 

OSS&E 

Level  2 

OSS&E 

Level  3 

OSS&E 

Level  4 

OSS&E 

Level  5 

OSS&E 

Level  6 

i. 

Introduction 

u 

Program  Description  and  Applicable  Documents 

u.i 

Program  Description 

Y 

1.1.2 

Applicable  Documents 

1.2 

Program  Technical  Status  as  of  Date  of  This  SEP 

Y 

1.3 

Approach  for  SEP  Updates 

2. 

Systems  Engineering  Application  to  Life  Cycle  Phases 

2.1 

System  Capabilities,  Requirements,  and  Design 
Considerations 

2.1.1 

Capabilities  to  be  Achieved 

2.1.2 

Key  Performance  Parameters 

Y 

G 

2.1.3 

Statutory  and  Regulatory  Requirements 

2.1.4 

Certification  Requirements 

Y 

G 

2.1.5 

Design  Considerations 

2.2 

SE  Organizational  Integration  and  Technical  Authority 

2.2.1 

Organization  of  IPTs 

2.2.2 

Organizational  Responsibilities 

Y 

Y 

Y 

2.2.3 

Integration  of  SE  into  Program  IPTs 

Y 

Y 

Y 

2.2.4 

Technical  Staffing  and  Hiring  Plan 

Y 

2.3 

Systems  Engineering  Process 

2.3.1 

Process  Selection 

G 

2.3.2 

Process  Improvement 

Y 

Y 

2.3.3 

Tools  and  Resources 

Y 

2.3.4 

Approach  for  Trades 

Y 

Y 

2.4 

Technical  Management  and  Control 

2.4.1 

Technical  Baseline  Management  and  Control  (Strategy 
and  Approach) 

Y 

G 

2.4.2 

Technical  Review  Plan  (Strategy  and  Approach) 

Y 

Y 

Y 

2.5 

Integration  with  Overall  Program  Management  Control 
Efforts 

2.5.1 

Acquisition  Strategy 

2.5.2 

Risk  Management 

2.5.3 

Integrated  Master  Plan 

2.5.4 

Earned  Value  Management 

2.5.5 

Contract  Management 

Legend 


(3 

Y 


OSS&E  information  satisfies  SEP  paragraph 
OSS&E  information  partially  satisfies  SEP  paragraph 
OSS&E  information  doesn’t  help  satisfy  SEP  paragraph 
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2.2. 1.1  OSS&E  Baseline  Characteristic  and  Key  Performance  Parameters  (KPPs) 

SEP  paragraph  2.1.2,  Key  Performance  Parameters ,  can  be  fully  satisfied  using  a  combination  of  the 
OSS&E  Level  4  exit  criteria  requiring  the  OSS&E  baseline  characteristics  be  identified  and 
coordinated  with  the  system/end-item  user;  and  the  OSS&E  Level  5  exit  criteria  that  fielded  systems 
be  assessed  against  OSS&E  baseline  characteristics. 

The  agreed  to  baseline  characteristics  should  be  tied  to  the  KPPs  if  they  exist.  If  KPPs  don’t  already 
exist,  the  technical/performance  baseline  characteristics  should  be  suitable  to  use  instead  of  KPPs. 

Consider  presenting  the  information  in  a  table.  Table  3  provides  an  example. 

Table  3:  Sample  KPP  Table 


KPP/Baseline 

Characteristic 

Parameter 

Threshold 

Objective 

Document 

Reference 

Targeting  Error 

Arc  distance  of  allowable 
error  from  targeted 
coordinates  at  a  distance  of 
500,000  kilometers. 

<  0.052  arc  radians  (3 
arc  degrees)  using 
onboard  visual  sighting. 

<0.017  arc  radians  (1 
arc  degree)  using 
onboard  computing 
resources. 

CPD  Para  6.2.1 

Firing  Time 

Elapsed  time  from  firing 
order  to  weapon  firing. 

<  90  seconds  using 
manual  firing 
procedures. 

<15  seconds  using 
onboard  computing 
resources. 

CPD  Para  6.2.3 

Net  Readiness 

The  degree  to  which  net- 
centric  system  activities, 
controls,  and  information 
exchanges  will  satisfy 

Global  Information  Grid 
(GIG)  space  network 
interfaces. 

100%  of  the  system’s 
designated  enterprise- 
level  or  critical  net- 
centric  activities, 
controls,  and 
information  exchanges 
satisfy  Global 
Information  Grid  (GIG) 
space  network 
interfaces  or  approved 
waivers. 

100%  of  the  system’s 
net-centric  activities, 
controls,  and 
information  exchanges 
satisfy  Global 
Information  Grid  (GIG) 
space  network 
interfaces  or  approved 
waivers. 

CPD  Para  6.3.1 

Communications 

Interoperability 

Interoperability  with  joint, 
service,  and  threat 
communication  systems. 

The  ability  to  send  and 
receive  Imperial  Link- 
86. 

The  ability  to  send  and 
receive  Republic  Link- 
99. 

CPD  Para  6.3.2 

2.2. 1.2  Technical  Baseline  Management  and  Control 

SEP  paragraph  2.4.1,  Technical  Baseline  Management  and  Control,  can  be  fully  satisfied  using  a 
combination  of  the  OSS&E  Level  2  exit  criteria  requiring  a  configuration  control  processes  be 
established  and  documented;  and  the  OSS&E  Level  5  exit  criteria  that  fielded  system/end-item  data 
gathered.  These  two  exit  criteria  support  the  AFMCI  63-1201,  paragraph  3.10.1,  requirement  that 
the  Chief  Engineer/Lead  Engineer  to  be  responsible  for  system  or  end- item  configurations. 

Since  the  configuration  control  process  is  documented,  all  that  is  needed  in  the  SEP  is  a  summary  of 
the  process  (a  nice  graphic  would  be  helpful),  and  a  pointer  to  the  details  -  including  to  artifacts  that 
show  you  follow  the  process.  When  the  process  was  first  identified,  the  initial  set  of  documents  and 
other  products  that  fall  within  the  span  of  control  may  have  been  identified.  It  is  this  set  that  can 


8 


OSS&E  Planning  to  SEP  Gap  Analysis 
28  September  2006 

serve  as  the  first  listing  of  the  technical  baseline.  Once  the  fielded  system/end-item  assessment  was 
completed,  the  full  set  of  artifacts  constituting  the  technical  baseline  should  have  been  identified.  It 
is  this  full  set  that  should  be  identified  in  the  SEP. 


2.2.1.3  Certifications 

SEP  paragraph  2.1.4,  Certification  Requirements,  can  be  fully  satisfied  using  a  combination  of  the 
OSS&E  Level  3  exit  criteria  requiring  a  plan  for  achieving  and/or  maintaining  required 
certifications;  and  the  OSS&E  Level  6  exit  criteria  requiring  all  required  certifications  be  in  place 
and  maintained. 

When  planning  to  achieve/maintain  certifications,  a  list  of  required  certifications  should  have  been 
generated.  This  list  forms  the  initial  listing  of  certifications  suitable  to  include  in  the  SEP  paragraph. 
Once  all  certifications  are  in  place,  a  simple  table  can  be  used  as  the  basis  for  the  SEP  paragraph. 
Table  4  provides  a  partial  example. 

Table  4:  Example  Certification  Table 


Certification 

Source/Reference 

Responsible  Person 

Completion 

Date(s) 

DoD  Information  Technology 
Security  Certification  and 
Accreditation 

DoDD  8500.1 

DoDI  5200.40 

DAG  7.5.10 

Electromagnetic 

Environmental  Effects  (E3) 
Control  and  Spectrum 
Certification 

CJCSM  3170.01 

CJCSI  6212.01 

DAG  1.63.1 

Information  Assurance 
Certification  and  Accreditation 

DAG  7. 2. 3.4 

Airworthiness  Certification 

MIL-HDBK-5 1 6A 

Global  Air  Traffic  Control 
Certification 

Joint  Interoperability 
Certification 

National  Security  Agency 
(NS A)  Cryptographic 
Certification 

Space  Flight  Worthiness 

2.2. 1.4  Systems  Engineering  Process 

SEP  paragraph  2.3.1,  Process  Selection,  can  be  fully  satisfied  using  the  OSS&E  Level  6  exit  criteria 
requiring  a  processes  be  established  and  in  place  to  maintain  OSS&E  baseline  characteristics.  This 
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process  is,  in  essence,  the  disciplined  (systems)  engineering  process  mentioned  in  AFMCI  63-1201, 
paragraph  3.10.1,  for  which  the  Chief  Engineer/Lead  Engineer  is  responsible  and  accountable  to 
apply. 

If  your  program  has  a  well  defined/documented  systems  engineering  process,  then  all  you  have  to  do 
is  explain  it  in  paragraph  2.3.1  of  a  sustainment  SEP.  If  your  program  doesn’t  have  a  well 
defined/documented  systems  engineering  process,  then  you  could  use  the  Defense  Acquisition 
Guidebook,  chapter  4,  as  a  guide.  In  either  case,  the  material  asked  for  in  the  Defense  Acquisition 
Guidebook,  chapter  4,  should  be  addressed  in  some  manner. 


Figure  2  shows  the  high-level  systems  engineering  process  from  the  Defense  Acquisition  Guidebook, 
chapter  4,  for  the  Operations  and  Support  Phase.  The  process  developed  for  your  program  need  not 
be  formatted  like  or  look  like  the  guidebook  process,  but  your  process  activities  should  be  able  to  be 
mapped  into  the  guidebook  process  activities. 


Operations  and  Support  Phase 


INPUTS 
•Sen/rce  Use 
•  Us  F&*db-xh 
rFi/Jiin  Roparti 
•SyjMm  Smtyty  s 

y  fle  ^  wti 

►50* _ 


OUTPUTS 


■  tapirt  toCPt?  format 
tnammut 

to  Utkivd  syttmm 

SEP 


_ 

Monitor  and  Co  Ho  ct 

All  SetVke 
Uw  om 


■  Tii-SocvN;#'  i 


Implement  and 
Field 


Andil yzo  Dita  to 
Determine 
Root  Came 


Risk  ot 

Imp  roved  System 


Dntnnlililn 

System  Ritk.1 
Hat  aid  s *v+flty 


Intoqrate 
Corre  dive  Action 


£11 


■  Process  Change  - 
H^r-dvmre^Support 
+  MaleifleK  hangs 


Figure  2:  Systems  Engineering  Activities  During  Operations  and  Support 
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2.2.2  Partial  Connections  -  Filling  Gaps 

What  follows  is  the  reasoning  behind  the  conclusion  that  an  OSS&E  Level  6  program  can  partially 
satisfy  nine  SEP  paragraphs:  1.1.1,  Program  Description ;  1 .2,  Program  Technical  Status',  2.2.2, 
Organizational  Responsibilities',  2.2.3,  Integration  of  Systems  Engineering  into  Program  IPTs ;  2.2.4, 
Technical  Staffing  and  Hiring  Plan',  2.3.2,  Process  Improvement',  2.3.3,  Tools  and  Resources',  2.3.4, 
Approach  for  Trades',  and  2.4.2,  Technical  Review  Plan.  Again,  each  program  is  free  to  implement  and 
document  OSS&E  as  it  makes  sense  for  the  program,  so  specific  OSS&E  artifacts  are  not  mentioned. 
This  section  cross  references  the  applicable  OSS&E  level  exit  criteria  for  which  useful  artifacts  should 
have  been  developed  and  the  SEP  paragraphs  they  may  apply  to. 

2.2.2. 1  Program  Description 

SEP  paragraph  1.1.1,  Program  Description,  can  be  partially  satisfied  using  the  OSS&E  Level  1  exit 
criteria  of  having  the  system/end-item  on  the  system/end-item  list  and  having  a  process  in  place  to 
update  the  list.  This  should  provide  at  least  a  short,  top-level  system  description  of  the  program. 

Some  recommendations  to  help  fill  the  gap  include: 

a.  Expand  the  top-level  system  description  to  assure  the  overall  key  aspects  of  the  program  are 
conveyed. 

b.  Use  a  graphic  or  picture  such  as  a  High  Level  Operational  Concept  Graphic  (DoDAF  OV-1) 
especially  if  it  shows  any  family-of-systems  (FoS)  and/or  system-of-systems  (SoS) 
relationships.  Figure  3  provides  an  example. 


Figure  3:  Example  High  Level  Operational  Concept  Graphic 
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2.2.2.2  Technical  Status 

SEP  paragraph  1.2,  Program  Technical  Status,  can  be  partially  satisfied  using  the  OSS&E  Level  5 
exit  criteria  of  assessing  fielded  systems/end-items  against  the  OSS&E  baseline  criteria.  This  should 
give  an  overall  picture  of  system/end-item  health  as  well  as  insight  to  the  overall  adequacy  of  the 
technical  baseline.  This  information  should  be  summarized  within  the  paragraph. 

Some  recommendations  to  help  fill  the  gap  include: 

a.  Include  any  current/upcoming  milestone  information.  In  a  sustainment  SEP,  you  should 
already  be  passed  the  designated  acquisition  milestones,  so  these  milestones  should  track  to 
major  program  reviews  or,  perhaps,  to  decision  points  where  system  modification  and 
modification  approval  are  considered.  Once  a  modification  project  is  approved,  that  project 
should  then  create  a  development  SEP  that  ties  into  the  sustainment  SEP. 

b.  Summary  of  past  milestones  achieved.  There  is  no  need  for  excruciating  detail  or  verbosity 
here.  The  summary  for  each  milestone  should  include  what  the  stated  exit  conditions  were 
and  how  well  they  were  or  weren’t  met.  For  each  criterion  not  met,  a  short  explanation 
should  be  included. 

c.  Identify  any  critical  path  and  tracking  event.  This  may  not  be  applicable  to  a  sustainment 
SEP,  but  will  apply  to  a  modification  SEP. 

d.  Identify  open  hazards. 

e.  Summarize  the  status  of  deliverables  or  key  events  required  by  other  programs  in  order  to 
field  and  sustain  a  complete,  FoS  or  SoS  mission  capability,  if  applicable. 

f.  Summarize  the  OSS&E  effort  and  tie  it  in  to  the  overall  program  and  systems  engineering. 

2.2.2.3  Organizations 

SEP  paragraph  2.2.2,  Organizational  Responsibilities,  can  be  partially  satisfied  using  the  OSS&E 
Level  1  exit  criteria  requiring  a  Chief  Engineer  be  assigned,  the  Level  3  exit  criteria  requiring  the 
OSS&E  effort  be  planned  and  documented,  and  the  Level  4  criteria  requiring  system/end-item  user 
coordination.  Each  of  these  criteria  requires  a  person  or  people  from  one  or  more  organizations  to 
actually  do  something.  Much  of  this  should  already  have  been  captured  in  the  OSS&E  Plan  and 
should  be  readily  transferable  to  the  SEP. 

Some  recommendations  to  help  fill  the  gap  include: 

a.  Identify/list  and  summarize  all  the  OSS&E  MOAs/MOUs.  Program  offices  may  have 
MOAs/MOUs  with  organizations  such  as:  the  Defense  Logistics  Agency  (DLA);  AFMC 
logistics,  test,  or  product  centers;  the  Navy;  the  Army;  operational  commands;  the  Air  Force 
Reserve;  the  Air  National  Guard;  etc. 

b.  Use  a  graphic  or  picture  such  as  an  Organization  Relationships  Chart  (DoDAF  OV-4)  to 
show  organizational  hierarchies  and  relationships. 
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c.  Be  specific  as  to  what  organization  has  what  responsibilities.  Refer  to  actual  people  and  tie 
the  organizations  into  the  IPT  structure  (SEP  paragraph  2.2.3,  Integration  of  Systems 
Engineering  into  Program  IPTs ). 

2.2.2.4  Integrated  Product  Teams  (IPTs) 

SEP  paragraph  2.2.3,  Integration  of  Systems  Engineering  into  Program  IPTs,  can  be  partially 
satisfied  using  the  OSS&E  Level  2  exit  criteria  to  identify  and  document  delegated  authority,  the 
Level  3  exit  criteria  to  coordinate  with  the  OSS&E  plan  with  the  users,  and  the  Level  4  exit  criteria 
to  coordinate  the  OSS&E  baseline  characteristics  and  metrics  with  the  users.  In  each  of  these  cases, 
a  group  of  people  should  have  been  identified  to  accept  delegated  authority  and  to  handle  the 
coordination.  Those  people  delegated  authority  may  be  the  equivalent  of  IPT  leads  and  those 
handling  the  coordination  may  represent  an  IPT.  These  things  already  taken  care  of  through  OSS&E 
can  form  the  basis  to  start  satisfying  this  SEP  paragraph. 

Some  recommendations  to  help  fill  the  gap  include: 

a.  Graphically  show  the  program’s  IPT  structure  in  SEP  paragraph  2.2.1,  Organization  of  IPTs. 
Include  references  to  the  Work  Breakdown  Structure  (WBS)  elements  and  the  specifications 
each  individual  IPT  oversees.  Figure  4  provides  an  example. 


Figure  4:  Example  IPT  Chart  Incorporating  WBS  and  Specification  Tree  Information 
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Identify/highlight  any  IPT  that  has  a  systems  engineering  presence  and  explain  the  systems 
engineering  roles  and  responsibilities  within  those  IPTs. 

If  systems  engineering  is  a  separate  IPT,  explain  how  that  IPT  is  integrated  with  other  IPTs 
to  show  how  systems  engineering  works  with  and  influences  the  other  IPTs  to  assure  a 
disciplined  systems  engineering  process  is  followed. 

2.2.2.5  Staffing 

SEP  paragraph  2.2.4,  Technical  Staffing  and  Hiring  Plan,  can  be  partially  satisfied  using  the  OSS&E 
Level  2  exit  criteria  to  identify  configuration  control  process  training  requirements.  This 
information  can  be  included  in  the  staffing  and  hiring  plan  as  required  training  for  all  new  engineers 
and  project  managers  upon  being  hired. 

Some  recommendations  to  help  fill  the  gap  include: 

a.  Identify  key  positions  within  the  systems  engineering  organization  and  provide  the 
required/desired  experience,  skills,  and  knowledge  need  to  perform  within  those  positions. 
Include  possible  training  opportunities  to  upgrade  individuals  who  don’t  meet  all  the 
qualifications. 

b.  Identify  general  engineering  and  staff  positions  and  provide  the  required/desired  experience, 
skills,  and  knowledge  need  to  perform  within  those  positions.  Include  possible  training 
opportunities  to  upgrade  individuals  who  don’t  meet  all  the  qualifications. 

c.  Provide  a  comparison  between  what  a  fully  staffed  systems  engineering  function  would  look 
like  against  what  really  exists.  If  gaps  in  the  systems  engineering  organization  exist,  explain 
how  full  staffing  will  be  attained,  why  full  staffing  will  not  be  attained,  or  why  full  staffing  is 
not  needed. 

d.  Provide  a  reference/link  to  any  center-level  or  general  staffing  plan  that  may  have  relevant 
information. 

2.2.2.6  Systems  Engineering  Process  Improvement 

SEP  paragraph  2.3.2,  Process  Improvement,  can  be  partially  satisfied  using  the  OSS&E  Level  3  exit 
criteria  to  plan  for  data  system  feedback  mechanisms  and  Level  6  exit  criteria  to  have  established 
feedback  mechanisms  to  monitor  OSS&E  health.  The  results  of  the  feedback  data  analysis  may  be 
useful  in  assessing  the  effectiveness  of  the  systems  engineering  process  and  identifying  potential 
process  improvements.  If  this  is  done,  it  should  be  explained  within  this  SEP  paragraph. 

Some  recommendations  to  help  fill  the  gap  include: 

a.  Explain  how  process  effectiveness  is  measured/evaluated  and  who  is  responsible  for 
improving  the  process. 

b.  Include  a  summary  of  any  Air  Force  Smart  Operations  21  (AFS021)  initiatives  both  being 
implemented  and  being  planned. 
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c.  Identify  who  has  the  authority  to  approve  process  changes  and  how  those  changes  are 
managed  and  controlled. 

2.2.2.1  Tools  and  Resources 

SEP  paragraph  2.3.3,  Tools  and  Resources,  can  be  partially  satisfied  using  the  OSS&E  Level  6  exit 
criteria  to  have  established  feedback  mechanisms  to  monitor  OSS&E  health.  What  tools  are  used  to 
get  and  analyze  the  feedback  should  form  the  beginning  of  a  tools  list. 

Some  recommendations  to  help  fill  the  gap  include: 

a.  Identify  and  list  the  tools  used  to  automate  the  systems  engineering  process,  to  manage 

requirements  and  system  configuration,  to  develop  and  sustain  software  and  hardware,  to  test 
and  evaluate  system  performance,  to  budget  for  future  systems  engineering  efforts,  and  to 
provide  training.  Some  examples/ideas  are  included  in  Table  5. 

Table  5:  Example  Tools/Resources  Listing 


Tool/Resource 

Purpose 

Owner 

Users 

Integrated  Digital  Environment 

Information  Resource  Management 

Dynamic  Object-Oriented 

Requirements  System  (DOORS) 

Electronic  Change  Request  System 

Digital  Image  Management  System 

Engineering  Source  Data  Requirements 

Data  Library 

Manufacturing  Resource  Planning  and 
Shop  Floor  Control 

Integrated  Master  Plan 

Integrated  Master  Schedule 

Risk  Management  Board 

Earned  Value  Management  System 

Health  Visibility  Management  System 

b.  Relate  the  purpose  of  each  tool  back  to  one  or  more  systems  engineering  activities  to  show 
how  the  tool  supports  the  program’s  systems  engineering  effort. 

2.2.2.8  Trade  Analysis 

SEP  paragraph  2.3.4,  Approach  for  Trades,  can  be  partially  satisfied  using  the  OSS&E  Level  5  exit 
criteria  to  identify  OSS&E  baseline  characteristics  disconnects  and  recommend  corrective  actions 
and  the  Level  6  criteria  to  monitor  OSS&E  health.  In  these  cases,  decisions  have  already  had  to  be 
made  and  alternatives  have  been  evaluated.  You  can  get  a  good  start  on  this  SEP  paragraph  by 
identifying  the  types  of  decisions  made,  who  made  them,  and  how  they  were  made.  If  the  trade 
analysis  process  was  effective,  why  not  continue  to  use  it? 


15 


OSS&E  Planning  to  SEP  Gap  Analysis 
28  September  2006 


During  sustainment,  trade  analysis/studies  may  lead  to  system  modifications. 

Some  recommendations  to  help  fill  the  gap  include: 

a.  Describe  any  analysis/studies  planned  for  making  trades  among:  stated  requirements;  design; 
project  schedule;  functional  and  performance  requirements;  function;  task;  and  decision 
allocation  among  human,  software,  and  hardware  and  life  cycle  and  design  to  cost. 

b.  Describe  the  trade  analysis/study  process  and  methods  to  be  used. 

c.  Include  the  intended  measures  of  effectiveness  (MOE)  and  how  they  interrelate. 

d.  Include  criteria  for  the  selection  of  measures  of  performance  (MOP)  to  support  the  evolving 
definition  and  verification  of  the  system  including  how  they  support  the  MOEs. 

e.  Include  how  the  analytical  results  are  integrated  and  the  criteria  used;  rationale  for  the 
solution;  evaluation  of  ESOR  hazards,  mitigation  and/or  associated  formal  risk  acceptance; 
and  how  performance  requirements,  life  cycle  costs,  etc,  will  be  considered. 

f.  Summarize  recent  trade  analysis/studies  and  how  they  have  steered  the  technical  and 
programmatic  changes  to  the  program. 

2.2.2.9  Technical  Reviews 

SEP  paragraph  2.4.2,  Technical  Review  Plan,  can  be  partially  satisfied  using  the  OSS&E  Level  3 
exit  criteria  to  establish  metrics;  the  Level  4  exit  criteria  to  measure  safety,  suitability,  and 
effectiveness;  and  the  Level  5  exit  criteria  to  recommend  corrective  actions  to  users.  For  each  of 
these  criteria  to  be  met,  some  sort  of  technical  interchange  had  to  take  place.  The  interchanges  may 
not  have  been  identified  as  formally  recognized  acquisition  related  technical  reviews,  but  technical 
reviews  they  were.  By  documenting  how  they  occurred,  who  participated,  and  what  the  results  were, 
you’ll  have  a  good  start  to  the  intent  of  this  SEP  paragraph. 

Some  recommendations  to  help  fill  the  gap  include: 

a.  Don’t  worry  about  minimizing  future  technical  reviews,  just  identify  and  plan  those  reviews 
that  the  program  needs.  Seasoned  travelers  know  that  shortcuts  frequently  take  more  time 
and  are  more  expensive  than  the  beaten  path. 

b.  Emphasize  that  your  technical  reviews  are  event  driven.  Show  that  the  program  knows  a 
predetermined  date  is  not  an  event  in  and  of  itself. 

c.  Use  table  6,  Generic  Technical  Review  Template,  as  a  template.  The  amount  of  information 
included  in  table  6  may  appear  overwhelming  because  there’s  a  lot  of  it.  Just  take  a  look  at 
the  template  and  use  it  as  something  to  help  you  consider  your  options.  The  technical 
reviews  are  yours  and  should  reflect  and  meet  the  needs  of  the  program. 
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Table  6:  Generic  Technical  Review  Template 


Review  Name 

What  is  the  name  of  this  review? 

Purpose 

Why  are  we  having  this  review? 

Statements  like,  “Because  it’s  required”,  are  not  good  purposes. 

Participants 

Chair 

—  All  participants  should  be  identified  by  name,  functional  title,  and  office  — 
This  can  be  someone  from  within  the  program,  but  could  also  be  the  Independent 
Subject  Matter  Expert  (SME). 

Independent  SME 
Stakeholders 

This  is  someone  from  outside  the  program. 

User/Operator  Safety  Logistics  Depot 

Requirements  Generators  Training  Maintenance  Manpower 

Human  Systems  Vehicles  Power  Generation  Fuels 

Program  Manager/Director  Suppliers  Contractor  Subcontractor 

Systems  Engineering  DCMA  Contracting 

System  Sustainment  Manager  Other  guests 

Entrance  Criteria/ 
Event  Timing 

What  other  program  activities  must  be  completed  before  calling  this  review? 

What  technical  efforts/documents/drawings/funding  are  required  to  accomplish 
this  review? 

What  is  the  needed  maturity  of  the  documents/drawings  (threshold/objective)? 

Who  has  the  decision  authority/responsibility  to  call  this  review? 

Review  Conduct 

What  are  the  ground  rules? 

Who  has  final  say  on  dispute  resolution? 

What  are  the  roles  and  responsibilities  of  the  participants? 

Where  will  the  review  be  held? 

Who  is  the  host? 

Who  will  take  minutes?  Publish  minutes?  Approve  minutes? 

What  is  the  planned  agenda? 

How  will  everyone  know  when  the  review  is  completed? 

What  will  happen  if  the  review  can ’t  be  completed  successfully? 

Success  Criteria/ 

Key  Metrics 

What  is  the  measurable  definition  of  success  (threshold/objective)? 

What  is  the  political  definition  of  success?  Is  it  compatible  with  the  above? 

What  agreements  must  be  made  and/or  consensus  reached? 

Who  can  declare  success? 

Technical  Maturity 
Assessment 

How  will  the  information  considered  during  the  review  be  used  to  assess  the 
program ’s  technical  maturity? 

What  part  of  the  technical  baseline  is  being  assessed? 

2.2.3  No  Connections  -  Filling  More  Gaps 

2.2.3. 1  References 

To  satisfy  SEP  paragraph  1.1.2,  Applicable  Documents,  consider  the  following: 

a.  Identify  reference  documents  and  a  point  of  contact  for  each  document.  Consider  using  a 
simple  table.  Table  7  provides  an  example: 
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Table  7:  SAMPLE  Document  Listing 


Document  Title 

Document  OPR  (Name  and  Office) 

Document 

Date 

Analysis  of  Alternatives  -  Space  Based 
Stand-off  Attack  Capability 

CAPT  Grant  M.  Piece 

US  Space  Defense  Command/J6D 

23  Sep  10 

Capability  Development  Document  (CDD) 

-  SAMPLE  System 

18  Sep  13 

Concept  of  Operations  -  Space  Based 
Stand-off  Attack  Capability 

07  Nov  09 

Enabling  Concept  -  SAMPLE  System 

18  Oct  10 

Initial  Capabilities  Document  (ICD)  - 
SAMPLE  System 

21  Jan  11 

SAMPLE  Acquisition  Strategy 

16  Nov  10 

SAMPLE  Integrated  Master  Plan  (IMP) 

12  Sep  13 

SAMPLE  Integrated  Master  Schedule 
(IMS) 

12  Sep  13 

SAMPLE  Integrated  Risk  Management  Plan 

12  Sep  13 

SAMPLE  System  Program  Management 

Plan 

12  Sep  13 

SAMPLE  Work  Breakdown  Structure 
(WBS) 

12  Sep  13 

b.  Summarize  important  program  documents  and  reference  (provide  links)  to  the  sections  and 
pages  of  documents  that  contain  the  detailed  information. 

c.  Provide  the  hierarchy  of  these  documents.  Consider  doing  this  graphically. 

2.2.3.2  SEP  Updates 

To  satisfy  SEP  paragraph  1.3,  Approach  for  SEP  Updates,  consider  the  following: 

a.  Identify  the  events  that  trigger  SEP  updates  and  the  sources  of  those  updates.  Consider  using 
a  table  as  shown  in  table  8. 

b.  List  previous  SEP  submittals  by  date  as  part  of  a  change  log  table. 
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SEP  Update  Triggers 

Update  Source(s) 

Annual  Systems  Engineering  Review 

Program  Chief  Engineer  (government) 

Program  Chief  Engineer  (contractor) 

Initiation  of  a  Modification  Project 

Program  Chief  Engineer  (government) 

Program  Chief  Engineer  (contractor) 

Milestone  Decision  Point 

Milestone  Decision  Authority 

Center  Chief  Engineer 

Program  Chief  Engineer  (government) 

Program  Chief  Engineer  (contractor) 

Decommissioning  Decision 

Program  Chief  Engineer  (government) 

Program  Chief  Engineer  (contractor) 

2.2.3.3  Capabilities 

To  satisfy  SEP  paragraph  2.1.1,  Capabilities  to  be  Achieved,  consider  the  following: 

a.  Rename  this  paragraph  to  Capabilities  to  Sustain.  This  should  be  a  mission  level 
introduction  to  SEP  paragraph  2.1.2,  Key  Performance  Parameters. 

b.  Summarize  the  required  capability  to  sustain.  This  should  be  found  in  the  Capability 
Production  Document  (CPD),  if  one  exists. 

c.  Summarize  the  approved  operational  concept. 

d.  Reference/link  to  the  appropriate  documents. 

2.2.3.4  Legal  Requirements 

To  satisfy  SEP  paragraph  2. 1 .3,  Statutory  and  Regulatory  Requirements,  consider  the  following: 

a.  Use  the  DoD  Instruction  5000.2  Information  Requirements  associated  with  Operations  and 
Support  as  summarized  in  the  Defense  Acquisition  Guidebook  as  a  starting  point  if  other 
program  documents  don’t  already  summarize  the  legal  requirements. 

b.  Consult  with  legal  acquisition  counsel. 

c.  Describe  the  plan  for  achieving  those  requirements,  including  the  applicable  approving 
authority. 

d.  Explain  any  MDA  tailoring  of  regulatory  program  information. 

e.  Use  tables  to  present  the  statutes  and  regulations.  Tables  9  and  10  provide  examples. 
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Statute 

Purpose/Information  Required 

Responsible  Person 

Completion 

Date(s) 

Public  Law  107-248,  Sec. 

8088(a)  [an  appropriations  act] 

Registration  of  mission-critical 
and  mission-essential 
information  systems 

Public  Law  106-398,  Section 

811,  Acquisition  and 
Management  of  Information 
Technology 

Registration  of  mission-critical 
and  mission-essential 
information  systems 

10  USC  2432,  Selected 
Acquisition  Reports 

Selected  Acquisition  Reports 

10  USC  2433,  Unit  Cost  Report 

Unit  Cost  Report 

Public  Law  103-160,  Sec.  220 
as  amended  by  Public  Law  103- 
337,  Sec.  214,  Electronic 

Warfare  (EW)  T&E 

EW  programs  on  OSD  T&E 
Oversight  List 

10  USC  2435 

Program  Deviation  Report 

10  USC  2399 

Operational  Test  Plan 

Table  10:  Example  Regulatory  References 


Regulation 

Information  Required 

Responsible  Person 

Completion 

Date(s) 

DoD  Instruction  5000.2 

Component  Cost  Analysis 

Cost  Analysis  Requirements 
Description 

Component  Live-Fire  Test  and 
Evaluation  Report 

Defense  Acquisition  Executive 
Summary 

OMB  Circular  A- 1 1 ,  Part  7 

Earned  Value  Management 
Systems 

Federal  Aviation  Regulation 

Part  25 

In-flight  icing  characteristics 
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2.2.3.5  Design  Considerations 

To  satisfy  SEP  paragraph  2.1.5,  Design  Considerations,  consider  the  following: 

a.  Rename  this  paragraph  to  Design  Considerations  to  Sustain. 

b.  Identify  the  design  constraints  that  would  apply  to  any  future  modification  effort. 

2.2.3.6  Integrated  Product  Teams  (IPTs) 

To  satisfy  SEP  paragraph  2.2.1,  Organization  of  IPTs,  consider  the  following: 

a.  Graphically  show  the  program’s  IPT  structure.  Include  references  to  the  Work  Breakdown 
Structure  (WBS)  elements  and  the  specifications  each  individual  IPT  oversees.  Figure  5 
provides  a  sample. 


Figure  5:  SAMPLE  IPT  Chart  with  WBS  and  Specification  Reference  Information 
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If  you  are  having  trouble  finding  a  documented  IPT  structure,  take  a  look  at  the  program 
telephone/contact  sheet.  Many  times  these  are  arranged  in  a  manner  that  reflects  the  way 
things  actually  get  done. 

2.2.3.7  Strategy 

To  satisfy  SEP  paragraph  2.5.1,  Acquisition  Strategy,  consider  the  following: 

a.  Rename  this  paragraph  to  Sustainment  Strategy. 

b.  Explain  how  the  program’s  selected  sustainment  strategy  is  based  on  the  technical 
understanding  of  the  system/end-item. 

c.  Address  potential  modifications  and  the  strategies  that  might  be  used  to  acquire  them.  In  a 
modification  SEP,  this  paragraph  would  return  to  Acquisition  Strategy. 

2.2.3.8  Risk 

To  satisfy  SEP  paragraph  2.5.2,  Risk  Management,  consider  the  following: 

a.  Show  the  linkages  between  the  technical  risk  assessment  and  mitigation  efforts  and  the 
overall  risk  management  process.  Reference/link  to  the  program  Risk  Management  Plan. 

b.  Show/describe  sample  risks.  Table  1 1  provides  a  format  example. 

Table  11:  Example  Risk  Exposure  Table 


Risk  -  Explanation 

Current 

Assessment 

Affect  on  Planning  Efforts 

On  Orbit  Attack  Exposure  - 
multiple  governments  and 
terrorist  organizations  have  a 
capability  to  attack  orbital  targets 

Green 

Both  government  and  contractor  systems 
engineers  modified  technical  and  engineering 
plans  to  reflect  a  change  in  assembly  orbit. 
Launch  plans  and  payload  manifests  were 
changed. 

Launch  Vehicle  Shortage  - 
multiple  launches  per  day  for 
many  consecutive  days  will  be 
needed.  This  exceeds  current 
inventory  capability. 

Yellow 

Contractor  systems  engineers  have  planned 
flexible  payload  configurations  to  take 
advantage  of  every  booster  type  used  by  the 
USAF,  NASA,  and  European  Space  Agency 
(ESA). 

Amorphous  Silicon  (Solar  Cell) 
Shortage  -  silicon  foundry 
capability  is  increasing,  but  not 
yet  sufficient  to  meet  projected 
program  solar  cell  needs. 

Yellow 

Contractor  engineering  plans  include  options 
to  increase  on  orbit,  fuel  cell  capabilities  to 
support  low  power,  environmental 
applications. 

c.  Show  how  systems  engineering  participates  in  risk  management.  Provide  an  example  of  risk 
identification,  assessment,  and  mitigation  to  show  you  can  do  risk  management. 
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2.2.3.9  Integrated  Master  Plan 

To  satisfy  SEP  paragraph  2.5.3,  Integrated  Master  Plan,  consider  the  following: 

a.  Show  how  the  technical  activities  are  integrated  into  the  overall  program  management  effort 
through  the  Program  Management  Plan  (PMP)  or  Integrated  Master  Plan  (IMP)  and 
Integrated  Master  Schedule  (IMS). 

b.  Explain  how  systems  engineering  influences  the  PMP  or  IMP  and  IMS. 

2.2.3.10  Earned  Value 

To  satisfy  SEP  paragraph  2.5.4,  Earned  Value  Management,  consider  the  following: 

a.  Consider  changing  the  paragraph  name  to  Value  Management. 

b.  Explain  how  value  is  applied  to  sustainment  efforts  earned.  An  Earned  Value  Management 
System  (EVMS)  may  already  be  applied  to  a  sustainment  contract,  but  an  explanation  of  how 
value  is  earned  from  the  organic  (government)  effort  is  determined  and  used  should  be 
included. 

c.  Describe  the  technical  efforts  that  are  included  in  measuring  earned  value  and  how  earned 
value  is  mapped  to  the  technical  reviews. 

2.2.3.11  Contracts 

To  satisfy  SEP  paragraph  2.5.5,  Contract  Management,  consider  the  following: 

a.  Describe  how  the  contract,  subcontract,  and  supplier,  if  applicable,  technical  efforts  are 
managed. 

b.  Describe  how  sources  are  selected. 

c.  Describe  the  approach  for  contractor  award  fees  and  performance  incentives  and  what  are  the 
specific  incentives  for  systems  engineering. 

d.  Describe  the  contracting  strategies  for  incentivizing  contractors  to  design  for  optimum 
materiel  readiness  at  minimum  life-cycle  cost  (e.g.,  design  for  reliability  and  maintainability, 
or  design  for  corrosion  resistance). 

e.  Use  the  DoD  Guide  for  Contracting  for  Systems  Engineering  as  a  guide  when  it  becomes 
available. 
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This  page  was  intentionally  left  blank. 
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Appendix  A:  Abbreviations 


AFI 

AFIT 

AFMC 

AFMCI 

AFPD 

AFS021 

AT&F 

CCA 

CJCSI 

CJCSM 

CPD 

CSE 

CTP 

DAG 

DCMA 

DoDAF 

DoDD 

DoDI 

EW 

EVMS 

FoS 

HQ 

IMP 

IMS 

INCOSE 

IPT 

KPP 

MAIS 

MDA 

MDAP 

MOA 

MOU 


Air  Force  Instruction 

Air  Force  Institute  of  Technology 

Air  Force  Material  Command 

Air  Force  Material  Command  Instruction 

Air  Force  Policy  Directive 

Air  Force  Smart  Operations  21 

Acquisition,  Technology,  and  Fogistics 

Clinger  Cohen  Act 

Chairman  Joint  Chiefs  of  Staff  Instruction 

Chairman  Joint  Chiefs  of  Staff  Manual 

Capabilities  Production  Document 

Air  Force  Center  for  Systems  Engineering  at  AFIT 

Critical  Technical  Parameter 

Defense  Acquisition  Guidebook 

Defense  Contract  Management  Agency 

Department  of  Defense  Architecture  Framework 

Department  of  Defense  Directive 

Department  of  Defense  Instruction 

Electronic  Warfare 

Earned  Value  Management  System 

Family  of  Systems 

Headquarters 

Integrated  Master  Plan 

Integrated  Master  Schedule 

International  Council  on  Systems  Engineering 

Integrated  Product  Team 

Key  Performance  Parameter 

Major  Automated  Information  Systems 

Milestone  Decision  Authority 

Major  Defense  Acquisition  Program 

Memorandum  of  Agreement 

Memorandum  of  Understanding 
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NS  A 

National  Security  Agency 

OPR 

Office  of  Primary  Responsibility 

OSS&E 

Operational  Safety,  Suitability,  and  Effectiveness 

OUSD 

Office  of  the  Undersecretary  of  Defense 

OV-1 

Operational  View  1 :  DoDAF  High  Level  Operational  Concept  Graphic 

OV-4 

Operational  View  4:  DoDAF  Organization  Relationships  Chart 

PEO 

Program  Executive  Officer 

PMP 

Program  Management  Plan 

S&EI 

System/End  Item 

SAF 

Secretary  of  the  Air  Force 

SAMPLE 

Synthetic  Aperture  Magnetically  Polarized  light  Emitter 

SE 

Systems  Engineering 

SEP 

Systems  Engineering  Plan 

SME 

Subject  Matter  Expert 

SoS 

System  of  Systems 

T&E 

Test  and  Evaluation 

WBS 

Work  Breakdown  Structure 
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Effectiveness,  05  April  2000 

[3]  OUSD(AT&L)  Defense  Systems,  Systems  Engineering  Plan  Preparation  Guide,  version  1.02, 
10  February  2006 

http://www.acq.osd.mil/se/publications/pig/set>  prepguide  vl  2.pdf 

[4]  SAF/AQX,  AFI  63-1101,  Modification  Management,  1 7  July  200 1 
http://www.e-publishing.af.mil/pubfiles/af/63/afi63-1101/afi63-1101.pdf 

[5]  SAF/AQR.  AFPD  63-12,  Assurance  of  Operational  Safety,  Suitability,  and  Effectiveness,  01 
February  2000 

http://www.e-publishing.af.mil/pubfiles/af/63/afpd63-12/afpd63-12.pdf 

[6]  SAF/AQR.  AFI  63-1201 ,  Assurance  of  Operational  Safety,  Suitability,  and  Effectiveness,  01 
February  2000  [Wholly  incorporated  in  the  updated  AFI  63-1201,  Systems  Engineering,  release 
date:  on  or  about  1  October  2006] 

http://www.e-publishing.af.mil/pubfiles/af/63/afi63-1201/afi63-1201.pdf 

[7]  SAF/AQR,  E-mail,  CSE  Sustainment  SEP  Guidance  Task  Request,  21  July  2006 

[8]  OUSD(AT&L),  Defense  Acquisition  Guidebook,  chapter  4 
http://akss.  dau.mil/dag/DoD5000.  asp?view=document 

[9]  OUSD(AT&L),  Integrated  Defense  Acquisition,  Technology,  and  Logistics  Life  Cycle 
Management  Framework,  version  5.2,  August  2005 
http://www.dau.mil/pubs/IDA/IDA  04.aspx 

[10]  Dennis  M.  Buede,  The  Engineering  Design  of  Systems,  2000,  Wiley  Interscience  Publication 

[11]  Charles  S.  Wasson,  System  Analysis,  Design,  and  Development,  2006,  Wiley  Interscience 
Publication 

[12]  International  Council  on  Systems  Engineering  (INCOSE),  Systems  Engineering  Handbook,  A 
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Appendix  C:  OSS&E  Level  to  SEP  Traceability  Matrix 

This  appendix  breaks  out  the  exit  criteria  from  each  of  the  six  levels  of  OSS&E  implementation  as 
defined  in  the  HQ  AFMC/DR/EN  Memo,  Execution  of  Air  Force  Polices  for  Assurance  of  Operational 
Safety,  Suitability,  and  Effectiveness  (OSS&E),  25  September  2000.  It  also  maps  that  breakout  to  SEP 
paragraphs  where  the  information  needed  to  meet  the  criteria  could  be  reused. 

Cl  OSS&E  Level  1  to  SEP  Traceability 

Table  Cl:  OSS&E  Level  1  Exit  Criteria  to  SEP  Tracing 


Designated  OSS&E  Level 
Reference  OSS&E  Execution 
Memo  (24  Sep  00) 

SEP  Preparation 
Guide,  ver  1.02 
Reference 

SEP  Paragraph  Reference  and  Comments 

Level  1  -  Chief  Engineer 
Assigned 

1.1-  System/End-Item  (S&EI) 
on  OSS&E  S&EI  List 

3.3.1 

1.1.1  Program  Description:  Provide  the  top  level 
System/End-Item  description  and  any  family-of- 
systems  (FoS)  or  system-of-systems  (SoS) 
relationships. 

1.2-  Chief  Engineer  identified 
on  OSS&E  S&EI  list 

3.3.1 

1.1.1  Program  Description:  Include  a  paragraph 
referencing  the  S&EI  list. 

3.4.2 

2.2.2  Organizational  Responsibilities:  Identify  the 
chief  engineer,  by  name,  as  the  chief  technical 
authority  within  the  program  organization. 

1.3  -  Process  is  in  place  to 
update  S&EI  list 

3.3.1 

1.1.1  Program  Description:  Include  a  paragraph 
explaining  this  process  and  who  has  responsibilities 
to  assure  the  S&EI  listing  is  current. 
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C2  OSS&E  Level  2  to  SEP  Traceability 

Table  C2:  OSS&E  Level  2  Exit  Criteria  to  SEP  Tracing 


Designated  OSS&E  Level 
Reference  OSS&E  Execution 
Memo  (24  Sep  00) 

SEP  Preparation 
Guide,  ver  1.02 
Reference 

SEP  Paragraph  Reference  and  Comments 

Level  2  -  Configuration 

Control  Processes 
Established 

2. 1  -  Configuration  control 
processes  identified 
and  documented  at  the 
program  level 

3.4.4 

2.4.1  Technical  Baseline  Management  and  Control 
Strategy:  Summarize  the  process,  identify  the  role 
of  the  technical  authority,  identify  what  products  are 
managed,  and  reference  the  configuration  control 
process  document(s). 

2.2  -  Configuration  control 
process  training 
requirements 
identified 

3.4.2 

2.2.4  Technical  Staff  and  Hiring  Plan:  Include 
within  the  subsection  describing  “the  staffing  levels, 
training,  and  experience  needed  to  execute  the 
required  technical  effort”. 

2.3  -  Configuration  control 

processes  in-place  and 
operating 

3.4.4 

2.4.1  Technical  Baseline  Management  and  Control 
Strategy:  Identify  (list?)  what  products  are 
managed.  Summarize  the  process  and  reference  the 
document  that  describes/establishes  the  process. 

2.4  -  Delegated  authority 
identified  and 
documented 

3.4.2 

2.2.3  Integration  of  SE  into  Program  IPTs:  Show 
program  IPT  structure  and  where/how  systems 
engineering  fits  in.  Identify  who  participates  and 
what  their  role/authority  is. 

3.4.4 

2.4.1  Technical  Baseline  Management  and  Control 
Strategy:  Identify  at  what  level  the  products  are 
managed  and  who.  This  can  be  tied  back  in  to  the 

IPT  structure. 
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C3  OSS&E  Level  3  to  SEP  Traceability 

Table  C3:  OSS&E  Level  3  Exit  Criteria  to  SEP  Tracing 


Designated  OSS&E  Level 
Reference  OSS&E  Execution 
Memo  (24  Sep  00) 

SEP  Preparation 
Guide,  ver  1.02 
Reference 

SEP  Paragraph  Reference  and  Comments 

Level  3  -  Plan  to  Assure  and 
Preserve  OSS&E 
Documented 

3.1  -  Plan  shall  include 

strategies/ approach 
for: 

3.1.1  -  Identifying, 

reconciling,  and 
preserving  OSS&E 
baseline 
characteristics 

3.4.4 

2.4.2  Technical  Review  Plan  (Strategy  and 

Approach):  Explain  who,  how,  and  when 
(specifying  event  triggers). 

3.1.2  -  Achieving  and/or 
maintaining 
required 
certifications 

3.4.1 

2.1.4  Certification  Requirements:  Identify  the 
certifications  the  system  needs,  who  is  responsible  to 
assure  the  certification  is  accomplished/current,  and 
when  the  certification  is  do/was  accomplished. 

3.1.3  -  Establishing 

OSS&E  program 
level  and  product 
line  metrics 

3.4.4 

2.4.2  Technical  Review  Plan:  OSS&E  program 
level  and  product  line  metrics  may  be  applicable  as 
Critical  Technical  Parameters  (CTPs)  and  should  be 
tied  to  the  OSS&E  Baseline  Characteristics. 

3.1.4  -  Identifying  data 
system  feedback 
mechanisms 

3.4.3 

2.3.2  Process  Improvement:  Identify  what  feedback 
systems/data  you’re  using  and  how  the  information 
is  used  to  adjust  your  systems  engineering  effort. 
There  may  be  event  triggers  identified  that  set  off  an 
update  to  the  SEP. 

3.4.4 

2.4.2  Technical  Review  Plan  (Strategy  and 

Approach):  Explain  how  technical  reviews  are  used 
to  assess  both  system  and  process  maturity. 
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Table  C3:  OSS&E  Level  3  Exit  Criteria  to  SEP  Tracing  (continued) 


Designated  OSS&E  Level 
Reference  OSS&E  Execution 
Memo  (24  Sep  00) 

SEP  Preparation 
Guide,  ver  1.02 
Reference 

SEP  Paragraph  Reference  and  Comments 

3.2  -  OSS&E  Execution  Plan 
coordinated  with: 

3.2.1  -  Users 

3.4.2 

2.2.2  Organizational  Responsibilities:  What  user 
organizations  are  involved,  who  represents  them, 
what  are  their  roles,  and  what  are  their 
responsibilities.  Reference  any  MOA/MOU  with 
user  organizations. 

2.2.3  Integration  of  SE  into  Program  IPTs:  For  each 
IPT,  show  how  the  users  fit  in  and  how  they  relate  to 
systems  engineering. 

3.2.2  -  Appropriate 

Product,  Logistic, 
Test  and  Specialty 
Centers 

3.4.2 

2.2.2  Organizational  Responsibilities:  What  center 
(AFMC  or  other)  organizations  are  involved,  who 
represents  them,  what  are  their  roles,  and  what  are 
their  responsibilities.  Reference  any  MOA/MOU 
with  these  organizations. 

2.2.3  Integration  of  SE  into  Program  IPTs:  For  each 
IPT,  show  how  the  each  of  the  centers  fit  in  and  how 
that  relates  to  systems  engineering. 
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C4  OSS&E  Level  4  to  SEP  Traceability 

Table  C4:  OSS&E  Level  4  Exit  Criteria  to  SEP  Tracing 


Designated  OSS&E  Level 
Reference  OSS&E  Execution 
Memo  (24  Sep  00) 

SEP  Preparation 
Guide,  ver  1.02 
Reference 

SEP  Paragraph  Reference  and  Comments 

Level  4  -  OSS&E  Baselines 
Developed  and 
Coordinated  with  User 

4.1  -  OSS&E  baseline 
characteristics 
identified 

3.4.1 

2. 1 .2  Key  Performance  Parameters:  If  formal  KPPs 
haven’t  been  identified,  state  so  and  use  the  OSS&E 
baseline  characteristics.  If  KPPs  have  been 
identified,  then  show  how  the  OSS&E  baseline 
characteristics  support  the  KPPs. 

4.2  -  Critical  characteristics 
for  measuring  safety, 
suitability,  and 
effectiveness  selected 

3.4.4 

2.4.2  Technical  Review  Plan:  These  can  be  used  as 
Critical  Technical  Parameters  (CTPs)  and  should  be 
tied  to  the  OSS&E  Baseline  Characteristics. 

4.3  -  OSS&E  baseline 

characteristics  and 
metrics  coordinated 
with  users 

3.4.2 

2.2.2  Organizational  Responsibilities:  Include  the 
organization  responsible  to  initiate  coordination  as 
well  as  the  organizations  coordinating  on  the 

OSS&E  baseline  characteristics. 

2.2.3  Integration  of  SE  into  Program  IPTs:  Identify 
any  IPT  that  is  responsible  for  baseline  characteristic 
oversight,  review,  coordination,  or  approval. 
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C5  OSS&E  Level  5  to  SEP  Traceability 

Table  C5:  OSS&E  Level  5  Exit  Criteria  to  SEP  Tracing 


Designated  OSS&E  Level 
Reference  OSS&E  Execution 
Memo  (24  Sep  00) 

SEP  Preparation 
Guide,  ver  1.02 
Reference 

SEP  Paragraph  Reference  and  Comments 

Level  5  -  OSS&E  Assessment 
of  Fielded 
Systems/End-Items 

5.1  -  Fielded  system/end-item 
data  gathered 

3.3.2 

1.2  Program  Technical  Status  as  of  Date  of  This 

SEP:  Identify  the  latest  status  of  the  Fielded 
system/end-item. 

3.4.4 

2.4.1  Technical  Baseline  Management  and  Control 
(Strategy  and  Approach):  Explain  how  field  data  is 
assessed  and  used  within  the  configuration  control 
process. 

5.2  -  OSS&E  baseline 
characteristics 
assessment  completed 

3.3.2 

1.2  Program  Technical  Status  as  of  Date  of  This 

SEP:  Identify  the  latest  status  of  the  baseline 
characteristics. 

3.4.1 

2.1.2  Key  Performance  Parameters:  Assessment 
may  result  in  changes  to  those  identified  at  level  4. 

If  formal  KPPs  haven’t  been  identified,  state  so  and 
use  the  OSS&E  baseline  characteristics.  If  KPPs 
have  been  identified,  then  show  how  the  OSS&E 
baseline  characteristics  support  the  KPPs. 
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Table  C5:  OSS&E  Level  5  Exit  Criteria  to  SEP  Tracing  (continued) 


Designated  OSS&E  Level 
Reference  OSS&E  Execution 
Memo  (24  Sep  00) 

SEP  Preparation 
Guide,  ver  1.02 
Reference 

SEP  Paragraph  Reference  and  Comments 

5.3  -  OSS&E  baseline 

disconnects  identified 

3.3.2 

1.2  Program  Technical  Status  as  of  Date  of  This 

SEP:  Identify  the  latest  status  of  the  baseline 
characteristics. 

3.4.3 

2.3.4  Approaches  for  Trades:  How  are  the 
disconnects  used  to  help  decide  what  is  important 
when  determining  sustainment  efforts. 

3.4.4 

2.4.2  Technical  Review  Plan  (Strategy  and 

Approach):  Describe  any  technical  review(s)  used  to 
assess  the  baseline  characteristics  and/or 
disconnects. 

5.4  -  Recommended 

corrective  actions  to 

users 

3.4.3 

2.3.4  Approaches  for  Trades:  How  are  the 
recommended  corrective  actions  used  to  help  decide 
what  is  important  when  determining  sustainment 
efforts. 

3.4.4 

2.4.2  Technical  Review  Plan  (Strategy  and 

Approach): 
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C6  OSS&E  Level  6  to  SEP  Traceability 

Table  C6:  OSS&E  Level  6  Exit  Criteria  to  SEP  Tracing 


Designated  OSS&E  Level 
Reference  OSS&E  Execution 
Memo  (24  Sep  00) 

SEP  Preparation 
Guide,  ver  1.02 
Reference 

SEP  Paragraph  Reference  and  Comments 

Level  6  -  Full  OSS&E  Policy 
Compliance 

6.1-  All  required 

certifications  in  place 
and  maintained 

3.4.1 

2. 1 .4  Certification  Requirements:  This  should  result 
in  a  direct  listing  of  certification  requirements 
without  having  to  invent  any  additional  information. 
Identify  the  certifications  the  system  needs,  who  is 
responsible  to  assure  the  certification  is 
accomplished/current,  and  when  the  certification  is 
do/was  accomplished. 

6.2  -  Metrics  and  feedback 
systems  monitoring 
OSS&E  health 

3.4.3 

2.3.3  Tools  and  Resources:  List  the  tools  and  other 
resources  being  used,  who  is  responsible  for  them, 
the  purpose  for  each  tool/resource,  and  what  each 
one  is  for  (how  is  it  used). 

2.3.4  Approaches  for  Trades:  How  is  the  feedback 
information  incorporated  with  deciding  what  is 
important  when  determining  sustainment  efforts. 

6.3  -  Processes  established 
and  in  place  to 
maintain  OSS&E 
baseline  characteristics 

3.4.3 

2.3.1  Process  Selection:  Describe  the  process  that  is 
already  established,  reference  documents  that 
contain  the  detail,  and  delineate  who  is  responsible 
for  what,  when. 

2.3.2  Process  Improvement:  Describe  how  process 
improvement  initiatives  are  part  of  the  process,  how 
process  improvement  is  accomplished,  and  who  has 
what  authority/responsibility. 
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Appendix  D:  OSS&E  Level  to  SEP  Paragraph  Mapping 

This  appendix  provides  a  quick  reference  to  show  in  which  SEP  paragraphs  information  generated  in 
support  of  exit  OSS&E  level  exit  criteria  can  be  reused.  Table  D  shows  the  high  level  mapping  and 
includes  only  those  SEP  paragraphs  affected.  Any  SEP  paragraph  not  referenced  is  not  a  candidate  for 
OSS&E  information  reuse. 

Table  D:  Overall  Mapping  of  OSS&E  Level  Requirements  to  Applicable  SEP  Paragraphs 


SEP 

Paragraph 

Number 

SEP  Paragraph  Title 

OSS&E 

Level  1 

OSS&E 

Level  2 

OSS&E 

Level  3 

OSS&E 

Level  4 

OSS&E 

Level  5 

OSS&E 

Level  6 

1. 

Introduction 

1.1 

Program  Description  and  Applicable  Documents 

1.1.1 

Program  Description 

Y 

1.2 

Program  Technical  Status  as  of  Date  of  This  SEP 

Y 

2. 

Systems  Engineering  Application  to  Life  Cycle  Phases 

2.1 

System  Capabilities,  Requirements,  and  Design 
Considerations 

2.1.2 

Key  Performance  Parameters 

Y 

G 

2.1.4 

Certification  Requirements 

Y 

G 

2.2 

SE  Organizational  Integration  and  Technical 

Authority 

2.2.2 

Organizational  Responsibilities 

Y 

Y 

Y 

2.2.3 

Integration  of  SE  into  Program  IPTs 

Y 

Y 

Y 

2.2.4 

Technical  Staffing  and  Hiring  Plan 

Y 

2.3 

Systems  Engineering  Process 

2.3.1 

Process  Selection 

Y 

G 

2.3.2 

Process  Improvement 

Y 

Y 

2.3.3 

Tools  and  Resources 

Y 

2.3.4 

Approach  for  Trades 

Y 

Y 

2.4 

Technical  Management  and  Control 

2.4.1 

Technical  Baseline  Management  and  Control 
(Strategy  and  Approach) 

Y 

G 

2.4.2 

Technical  Review  Plan  (Strategy  and  Approach) 

Y 

Y 

Y 

Legend 


G 

Y 


OSS&E  information  satisfies  SEP  paragraph 
OSS&E  information  partially  satisfies  SEP  paragraph 


37 


OSS&E  Planning  to  SEP  Gap  Analysis 
28  September  2006 


Dl.  OSS&E  Level  1  Reference  to  SEP  Paragraph  Mapping 

Table  Dl  shows  the  detailed  OSS&E  Level  1  exit  criteria  mapping  to  the  affected  SEP  paragraphs.  The 
OSS&E  Level  1  and  its  exit  criteria  references  are: 

Level  1  -  Chief  Engineer  Assigned 

1.1  -  System/End-Item  (S&EI)  on  OSS&E  S&EI  List 

1.2  -  Chief  Engineer  identified  on  OSS&E  S&EI  list 

1.3  -  Process  is  in  place  to  update  S&EI  list 

Table  Dl:  Detailed  Mapping  of  OSS&E  Level  1  Exit  Criteria  to  Applicable  SEP  Paragraphs 


SEP 

Paragraph 

Number 

SEP  Paragraph  Title 

Level  1 

Ref  1.1 

Level  1 

Ref  1.2 

Level  1 

Ref  1.3 

1. 

Introduction 

1.1 

Program  Description  and  Applicable  Documents 

1.1.1 

Program  Description 

Y 

Y 

Y 

2. 

Systems  Engineering  Application  to  Life  Cycle  Phases 

2.2 

SE  Organizational  Integration  and  Technical  Authority 

2.2.2 

Organizational  Responsibilities 

Y 

Legend 


G 

Y 


OSS&E  information  satisfies  SEP  paragraph 
OSS&E  information  partially  satisfies  SEP  paragraph 
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D2.  OSS&E  Level  2  Reference  to  SEP  Paragraph  Mapping 

Table  D2  shows  the  detailed  OSS&E  Level  2  exit  criteria  mapping  to  the  affected  SEP  paragraphs.  The 
OSS&E  Level  2  and  its  exit  criteria  references  are: 

Level  2  -  Configuration  Control  Processes  Established 

2.1-  Configuration  control  processes  identified  and  documented  at  the  program  level 

2.2  -  Configuration  control  process  training  requirements  identified 

2.3  -  Configuration  control  processes  in-place  and  operating 

2.4  -  Delegated  authority  identified  and  documented 

Table  D2:  Detailed  Mapping  of  OSS&E  Level  2  Exit  Criteria  to  Applicable  SEP  Paragraphs 


SEP 

Paragraph 

Number 

SEP  Paragraph  Title 

Level  2 

Ref  2.1 

Level  2 

Ref  2.2 

Level  2 

Ref  2.3 

Level  2 

Ref  2.4 

2. 

Systems  Engineering  Application  to  Life  Cycle  Phases 

2.2 

SE  Organizational  Integration  and  Technical  Authority 

2.2.3 

Integration  of  SE  into  Program  IPTs 

Y 

2.2.4 

Technical  Staffing  and  Hiring  Plan 

Y 

2.4 

Technical  Management  and  Control 

2.4.1 

Technical  Baseline  Management  and  Control  (Strategy  and 
Approach) 

Y 

Y 

Y 

Legend 


G 

Y 


OSS&E  information  satisfies  SEP  paragraph 
OSS&E  information  partially  satisfies  SEP  paragraph 
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D3.  OSS&E  Level  3  Reference  to  SEP  Paragraph  Mapping 

Table  D3  shows  the  detailed  OSS&E  Level  3  exit  criteria  mapping  to  the  affected  SEP  paragraphs.  The 
OSS&E  Level  3  and  its  exit  criteria  references  are: 

Level  3  -  Plan  to  Assure  and  Preserve  OSS&E  Documented 

3.1  -  Plan  shall  include  strategies/approach  for: 

3.1.1  -  Identifying,  reconciling,  and  preserving  OSS&E  baseline  characteristics 

3.1.2  -  Achieving  and/or  maintaining  required  certifications 

3.1.3  -  Establishing  OSS&E  program  level  and  product  line  metrics 

3.1.4  -  Identifying  data  system  feedback  mechanisms 

3.2  -  OSS&E  Execution  Plan  coordinated  with: 

3.2.1  -  Users 

3.2.2  -  Appropriate  Product,  Logistic,  Test  and  Specialty  Centers 


Table  D3:  Detailed  Mapping  of  OSS&E  Level  3  Exit  Criteria  to  Applicable  SEP  Paragraphs 


SEP 

Paragraph 

Number 

SEP  Paragraph  Title 

Level  3 

Ref  3.1.1 

Level  3 

Ref  3. 1.2 

Level  3 

Ref  3. 1.3 

Level  3 

Ref  3. 1.4 

Level  3 

Ref  3.2.1 

Level  3 

Ref  3.2.2 

2. 

Systems  Engineering  Application  to  Life  Cycle  Phases 

2.1 

System  Capabilities,  Requirements,  and  Design 
Considerations 

2.1.4 

Certification  Requirements 

Y 

2.2 

SE  Organizational  Integration  and  Technical 

Authority 

2.2.2 

Organizational  Responsibilities 

Y 

Y 

2.2.3 

Integration  of  SE  into  Program  IPTs 

Y 

Y 

2.3 

Systems  Engineering  Process 

2.3.2 

Process  Improvement 

Y 

2.4 

Technical  Management  and  Control 

2.4.2 

Technical  Review  Plan  (Strategy  and  Approach) 

Y 

Y 

Y 

Legend 


G 

Y 


OSS&E  information  satisfies  SEP  paragraph 
OSS&E  information  partially  satisfies  SEP  paragraph 
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D4.  OSS&E  Level  4  Reference  to  SEP  Paragraph  Mapping 

Table  D4  shows  the  detailed  OSS&E  Level  4  exit  criteria  mapping  to  the  affected  SEP  paragraphs.  The 
OSS&E  Level  4  and  its  exit  criteria  references  are: 

Level  4  -  OSS&E  Baselines  Developed  and  Coordinated  with  User 

4.1  -  OSS&E  baseline  characteristics  identified 

4.2  -  Critical  characteristics  for  measuring  safety,  suitability,  and  effectiveness  selected 

4.3  -  OSS&E  baseline  characteristics  and  metrics  coordinated  with  users 

Table  D4:  Detailed  Mapping  of  OSS&E  Level  4  Exit  Criteria  to  Applicable  SEP  Paragraphs 


SEP 

Paragraph 

Number 

SEP  Paragraph  Title 

Level  4 

Ref  4.1 

Level  4 

Ref  4.2 

Level  4 

Ref  4.3 

2. 

Systems  Engineering  Application  to  Life  Cycle  Phases 

2.1 

System  Capabilities,  Requirements,  and  Design  Considerations 

2.1.2 

Key  Performance  Parameters 

Y 

2.2 

SE  Organizational  Integration  and  Technical  Authority 

2.2.2 

Organizational  Responsibilities 

Y 

2.2.3 

Integration  of  SE  into  Program  IPTs 

Y 

2.4 

Technical  Management  and  Control 

2.4.2 

Technical  Review  Plan  (Strategy  and  Approach) 

Y 

Legend 


G 

Y 


OSS&E  information  satisfies  SEP  paragraph 
OSS&E  information  partially  satisfies  SEP  paragraph 


41 


OSS&E  Planning  to  SEP  Gap  Analysis 
28  September  2006 


D5.  OSS&E  Level  5  Reference  to  SEP  Paragraph  Mapping 

Table  D5  shows  the  detailed  OSS&E  Level  5  exit  criteria  mapping  to  the  affected  SEP  paragraphs.  The 
OSS&E  Level  5  and  its  exit  criteria  references  are: 

Level  5  -  OSS&E  Assessment  of  Fielded  Systems/End- Items 

5.1  -  Fielded  system/end-item  data  gathered 

5.2  -  OSS&E  baseline  characteristics  assessment  completed 

5.3  -  OSS&E  baseline  disconnects  identified 

5.4  -  Recommended  corrective  actions  to  users 

Table  D5:  Detailed  Mapping  of  OSS&E  Level  5  Exit  Criteria  to  Applicable  SEP  Paragraphs 


SEP 

Paragraph 

Number 

SEP  Paragraph  Title 

Level  5 

Ref  5.1 

Level  5 

Ref  5.2 

Level  5 

Ref  5.3 

Level  5 

Ref  5.4 

1. 

Introduction 

1.2 

Program  Technical  Status  as  of  Date  of  This  SEP 

Y 

Y 

Y 

2. 

Systems  Engineering  Application  to  Life  Cycle  Phases 

2.1 

System  Capabilities,  Requirements,  and  Design  Considerations 

2.1.2 

Key  Performance  Parameters 

G 

2.3 

Systems  Engineering  Process 

2.3.4 

Approach  for  Trades 

Y 

Y 

2.4 

Technical  Management  and  Control 

2.4.1 

Technical  Baseline  Management  and  Control  (Strategy  and 
Approach) 

G 

2.4.2 

Technical  Review  Plan  (Strategy  and  Approach) 

Y 

Y 

Legend 


G 

Y 


OSS&E  information  satisfies  SEP  paragraph 
OSS&E  information  partially  satisfies  SEP  paragraph 
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D6.  OSS&E  Level  6  Reference  to  SEP  Paragraph  Mapping 

Table  D6  shows  the  detailed  OSS&E  Level  6  exit  criteria  mapping  to  the  affected  SEP  paragraphs.  The 
OSS&E  Level  6  and  its  exit  criteria  references  are: 

Level  6  -  Full  OSS&E  Policy  Compliance 

6.1-  All  required  certifications  in  place  and  maintained 

6.2  -  Metrics  and  feedback  systems  monitoring  OSS&E  health 

6.3  -  Processes  established  and  in  place  to  maintain  OSS&E  baseline  characteristics 

Table  D6:  Detailed  Mapping  of  OSS&E  Level  6  Exit  Criteria  to  Applicable  SEP  Paragraphs 


SEP 

Paragraph 

Number 

SEP  Paragraph  Title 

Level  6 

Ref  6.1 

Level  6 

Ref  6.2 

Level  6 

Ref  6.3 

2. 

Systems  Engineering  Application  to  Life  Cycle  Phases 

2.1 

System  Capabilities,  Requirements,  and  Design  Considerations 

2.1.4 

Certification  Requirements 

G 

2.3 

Systems  Engineering  Process 

2.3.1 

Process  Selection 

G 

2.3.2 

Process  Improvement 

Y 

2.3.3 

Tools  and  Resources 

Y 

2.3.4 

Approach  for  Trades 

Y 

Legend 


G 

Y 


OSS&E  information  satisfies  SEP  paragraph 
OSS&E  information  partially  satisfies  SEP  paragraph 
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